Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optional shutdown delay to tasks #3043

Merged
merged 6 commits into from
Aug 17, 2017
Merged

Conversation

schmichael
Copy link
Member

Fixes #2441

Defaults to 0 (no delay) for backward compat and because this feature
should be opt-in.

Fixes #2441

Defaults to 0 (no delay) for backward compat and because this feature
should be opt-in.
}

// No shutdown escape hatch for this delay, so don't set it too high
task.ShutdownDelay = 5 * time.Second
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Way too long :) Try to avoid sleeping for any longer than a second.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're unable to check the Consul ops before the delay is up we'll get a false failure. Mind if I make it 2s to give Travis plenty of time? Since this test runs in parallel with the rest of client/ -- which takes ~50s total -- it should slow down the overall wallclock time.

@@ -1981,8 +1981,8 @@ func TestTaskGroupDiff(t *testing.T) {
{
Type: DiffTypeEdited,
Name: "Driver",
Old: "docker",
New: "exec",
Old: "1s",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not the intended change I don't think?

@@ -71,6 +71,12 @@ job "docs" {
[Consul][] for service discovery. Nomad automatically registers when a task
is started and de-registers it when the task dies.

- `shutdown_delay` `(string: "0s")` - Specifies the duration to wait when
killing a task between removing it from Consul and sending it a shutdown
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSON job spec as well

@@ -1171,6 +1171,12 @@ func (r *TaskRunner) run() {
interpTask := interpolateServices(r.envBuilder.Build(), r.task)
r.consul.RemoveTask(r.alloc.ID, interpTask)

// Delay actually killing the task if configured. See #244
if r.task.ShutdownDelay > 0 {
r.logger.Printf("[INFO] client: delaying shutdown of alloc %q task %q for %q", r.alloc.ID, r.task.Name, r.task.ShutdownDelay)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems more of a debug?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just worry people are going to forget they set a shutdown delay and wonder why Nomad seems to hang since there's no indication outside of looking at the job file that this task is blocked on a shutdown delay...

I'll make it DEBUG though... users stopping/updating jobs should be aware of the job's settings and behavior.

@schmichael schmichael merged commit f143501 into master Aug 17, 2017
@schmichael schmichael deleted the f-2441-shutdown-delay branch August 17, 2017 21:37
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants